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Abstract 

Using game and probability theories, I study the French popular 
game 421, a perfect information stochastic stage game. The prob- 
lem is to find strategies maximizing the probability of some expected 
utility. I only solve a player's round against providence, a problem 
of fate stochastic management: beyond the backward induction solu- 
tion, bounded complexity motivates heuristic policies. For a unique 
goal utility, a simple optimal policy, ratchet, is obtained. Its result 
probabilities are compiled and used, for arbitrary utilities, as the logic 
of goal identification policies. Various policies appear, close to hu- 
man behavior, and are exactly evaluated by solving the Kolmogorov 
equation. 

k. w.: stochastic management, Kolmogorov equation, bounded com- 
plexity, human behavior. 
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1 Aim and interest of the study 

Following I look for strategies, maximizing the probability of win, or 
some expected utility, in the game 421 combining chance and decision (see 
appendix 0) . 

By the way, or, indeed, by serendipity, I encounter the problem of fate 
stochastic management: optimizing today's decisions, with respect to a fu- 
ture utility, and in spite of tomorrow's odds. Such issues as what are the 
optimal policies, in what circumstances, and how much they demand on 
intellectual resources, can be resolved mathematically, suggesting that man- 
agement could be an exact science (as part of operations research). 

A lottery is not a game, in the sense of game theory, but a stochastic 
process (a sequence of random variables). Game theory treats classically 
multiple player decision games, the archetype of which is chess. A game 
in which players' fates depend on both chance and their decisions, like 421 
and backgammon ||2|, is a stochastic game 0. Chance makes decision more 
complex. For example, consider a variation on chess: the player proposes 
a list of n moves, the actual one being determined by casting dice, n = 1 
yields the standard pure decision game; n = ni, where ni is the number of 
possible moves, yields a pure chance game; n ^ ni/2 yields a game of chance 
and decision, more complex than the former and the latter. 

Game theory primarily focuses on the existence proofs for optimal strate- 
gies. However, "usable techniques for obtaining practical answers" also mat- 
ter i, §1.1]. Indeed, little can be done from existence without construction: 
this is the old debate around Zermelo's axiom of choice. Hence the interest 
of investigating, as in Church's thesis [|], calculability, the existence of an 
algorithmic solution. But even calculability may not be sufficient for actual 
computation. For example, consider again chess, a finite but very large game: 
the algorithmic solution provided by the Zermelo theorem ch. 6] is of no 
practical use (until the final moves), as noticed by |^, §11.4], as it exceeds the 
capacity of any computer. The study of finite games does not stop with Zer- 
melo theorem, and this is because of complexity boundedness. Algorithms 
shall be compared not only with respect to optimality (degree of completion 
of the task) but also complexity, using a bit of complexity theory 

An algorithm is characterized by its optimality, size and computing time 
on a given computer, specialized by high-level functions and data. The algo- 
rithm may be good or bad, short or long, fast or slow. The three qualities and 
quantities are not independent: the exchange of computing time against size 
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is the principle of data compression, the exchange of computing time against 
optimality is the principle of heuristics. When setting two quantities, the 
minimum of the third one, as a function of other implicit parameters, can 
be defined: the minimum size and the minimum computing time are respec- 
tively related with the Kolmogorov complexity and the Bennett logical depth 

§• 

For strategy-generating algorithms, or deciding algorithms, or policies, 
optimality is the expected utility. For a perfect information stage game, it is 
interesting, for possible extensions, to characterize the asymptotic behavior 
of the computing time, when the depth tends to infinity, i. e. to know 
whether the algorithm is linear or polynomial, rather than exponential, as 
feared from the tree structure of the game. 

The present study is thus an occasion to relate with each other, on a 
live case, various tools and concepts attached to games, processes, probabil- 
ities, control, programming, algorithmics and complexity, with applications 
in management and game practice. 

2 Backward induction optimal policy 

A stochastic game reduces to a pure decision game, by considering providence 
as a particular player p], ch. 4], whose mixed strategy, known a priori, results 
from usual statistical postulates (independence, stationarity) and cannot be 
optimized]] 421, thus reduced, and with some precautions on the rules (ap- 
pendix 0), is a perfect information finite game and the Zermelo theorem 
applies. 

I will solve only a sub-game, the player's round against providence (while 
other players stand still), a stochastic management problem, featuring a mar- 
tingale problem and, for the first player, a stopping time problem Q. The 
analogy with Brownian motion provides statistical mechanics tools. 

2.1 Alea 

Let D G N be the number of dice, normally 3, and F G N* the number of 
faces of every dice, normally 6. Dice are discernible|, so that the probability 

^Probability theory began as the study of the providential strategy in chance games, 
at the time of Bayes or the BernouUis. 

^Discernibility is not an innocuous hypothesis, as shown by Gibbs' paradox |[l0|| . 
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space is the set of face sequences, or arrangements. The class of arrange- 
ments corresponding to each other by a permutation is a combination, e. g., 
"nenette", 221, is the subset of arrangements {(1, 2, 2), (2, 1, 2), (2, 2, 1)}, of 
redundancy three. In 421, which dice produced which face does not matter, 
because ranking depends only on combination. 

I describe the die system as in statistical mechanics: each die is a parti- 
cle, with only one phase variable, face. The laws of mechanics are replaced 
by usual statistical hypothesis, abstracting chance from any specific random 
generator. The system is described, in Lagrangian notation, by a face com- 
bination, or, in Eulerian notation, by the sequence df of occupation numbers 
of every face / = 1 . . . F, e. g. the Lagrangian notation 421 translates into 
the Eulerian notation (1, 1, 0, 1, 0, 0) {F = 6). 

The interest of Eulerian notation lies in that the set of Eulerian combi- 
nations 

d = idf)f=i,„F. 

is the partially ordered normed vector space . The canonic basis = 
{Sf^g)f is aligned with "brelans", combinations with all faces of a kind. I 
define the ball 

B{D eN) = {de Z^, |d| < D} 

and similarly (replacing < above by = or < ) the sphere dB{D) and the 
open ball B{D). The intersections with the positive cone are represented 
by + exponents; the set of actual combinations is B^{D). The norm of a 
combination is the sum of Eulerian component absolute values. The norm 
of a positive combination is just its number of dice. The canonic order <, 
partial on 1/ , differs from the hierarchic order ^ (0), total on dB~^(D). 

Distinct casts are independent and the probability of any face to be on 
top is 1/F (unloaded dice). The arrangements of one combination are thus 
equiprobable, and the probability of a combination is just that of any of its 
arrangements, times the combination redundancy. For example, the prob- 
ability of obtaining the combination 21 is 2/F^, while the probability of 
obtaining the combination 11 is l/F"^. More generally, the probability of 
obtaining the combination d, after one cast, is given by the multinomial law, 
with usual notations generalizing power and factorial to integer vectors: 

P(d) = p'^^,P = }(l...l)GQ^ P(d) = l. (1) 

dedB+{D) 
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2.2 Fate 

For all j G N, let the state dj be the combination, accumulated after j casts, 
and the event dj+1/2 be the combination, obtained from the j + 1-th cast. 
Fate is the infinite state and event alternate sequence 

V9 = (do,di/2,di,d3/2...)- (2) 

The integer or half-integer index is used as a discrete time, integer time for 
states, half-integer for events. The set of possible fates is described by the 
fate tree, where branching represents chance (from integer time to half-integer 
time) or decision (conversely). 
The rules of 421 imply: 



do = 0, (3) 

VjG N,d,+i/2 G B+{D, = D-\d,\), (4) 

< dj+i - dj < dj+1/2, (5) 

3(jGN,j< J),d, G B+{D), (6) 



where J G N is the maximum round duration, normally 3. Dj is the number 
of live dice, which have not been accumulated after j events and one state. 
From (|,|),Do = A^J = 0. 
From (ID, 

Dhi < Dj, (7) 
(d,+i - d,- = d,+i/2) ^ d,+iGB+(D). (8) 

The effective round duration Ji is the minimum of j in (^. The next 
players' effective round durations must equal the first player's. Therefore, 
for all players, 

V(jGN,j< Ji),d, Gfi+(D) , d,+i/2 7^0, (9) 
dj,eB^{D) , (10) 
V(jGN,j> Ji),d,_i/2 = , d, =dj,. (11) 

(H, 0) are used, firstly, after the first player's end of round, to determine 
Ji, subsequently, as additional rules for next players. When j increases, the 
state vector dj moves in the positive ball, off the origin, towards its boundary 
where it gets stuck at dj^ , the round result. Fate is virtually continued by an 
infinite sequence, asymptotically alternating the result and the null event. 
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2.3 Utility 

Following von Neumann and Morgenstern ch. 27], a player's utility is a 
number, given by a causal function, i. e. a function of history (past fate), 
compatible with the player's preferences, and such that the utility before 
a random event is just the expected utility, i. e. the probability-weighted 
utility average, over possible outcomes. Thus, expected utility is anti-causal, 
i. e. prescribing utilities at some future time determines its expectation at 
all prior times. 

One never knows when a game actually stops, as a it is often embedded 
in a larger game. Tennis is a familiar example: a tennis "game" is a sub- 
game of a set, itself a sub-game of a match, tournament, ranking system . . . 
this cascade does not even stop with a player's life, because of cooperation 
between individuals. But, if we want to obtain any result, we must stop 
somewhere in the game cascade, and judge utility more or less empirically. 
(Quite similarly, in mechanics or thermodynamics, the studied system is 
coupled with the rest of the world, by an often delicate boundary or cut-off 
condition.) 

The study of 421 should stop at end of game, by setting players' utilities, 
for example, a binary utility: one for win, zero for loss, or incorporating 



economy, a la Bernoulli, the logarithm of earning divided by wealth |[Tl 
However, I treat only the round. At end of round, the Bernoulli formula does 
not make sense and utility is not given directly by the rules (in particular, 
the transfer function of table |^). By examining the rules, a few properties 
of utility are obtained; for example, at constant time, for a rational player, 
utility must be compatible with the hierarchic order ([5^) , etc. 

But I will not further characterize utility. On the contrary, I will consider 
the round independently of the rest of the game, with arbitrary utilities, in 
order to treat the problem of fate stochastic management in a rather general 
way. 

For all fate (H), utility is judged at some time J^, either integer or half- 
integer in general (in the round, G {1/2,3/2,5/2}), as a causal function: 

M(do,di/2...djJ gQ. (12) 

The function u has a variable number of arguments, formally, it is defined 
on UjeN* B^{Dy . Utility is judged forever: 

u{...dj^,.)^u{...dj^). (13) 
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The rules (^, ^, |^, |^, |T^) are superseded by — oo utilities for rule breaking 
histories (excluding cheating). In particular, the utilitarian version of (||) is 

V(j G N, d,+i/2 ^ B+iD,)), u{. . . d„ d,+i/2) = -oo (14) 



and the next players' round duration conditions (g, 0) become 



V(j eN,j< Ji,d, e dB+{D)),u{. . .d,) = -oo. (15) 
2.4 Optimal strategies 

The greatest utility, drawn from any event-terminated history, is 

Vj G N,m(. . .dj,dj+i/2) = maxM(. . .dj,dj+i/2,dj+i), (16) 

dj+i 

u{.) = maxM(.,di). (17) 

di 

The latter equation, where di is a dummy variable, is a more formal expres- 
sion of the former. The nature of the dummy variable is shown by its index 
(state for integer, event for half-integer). The set of states, corresponding to 
optimal decisions, is 

Su{-) = argmaxM(., di). (18) 

di 

A player's mixed strategy consists in choosing randomly between many 
decisions, according to a causal probability law, 

d ^ P(., d) ^ P(di = d\.),J2 Pi-^ di) = 1. (19) 

di 

V{X) means the probability of the event X. The optimal mixed strategies 
are such that the support of the probability law (|T9|) is a subset of Su{-) 
(among them are pure optimal strategies). 

From the von Neumann-Morgenstern theorem, 

«(.) = 5^p(.,di/2)^i(.,di/2). (20) 

di/2 

where j9 is a causal probability law, expressing the providential strategy and 
rules. Because of utility conditions, such as (|T^ , [T5|) , there are, in (pO|) , 
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products p X u of the undetermined form x oo, which ought to be replaced 
by zero (or the summation ought to be properly restricted). 
Combining (O, |20) , or conversely, 



niax^p(.,d3/2)u(.,di,d3/2), (21) 



^3/2 



u{.) = V'p(.,di/2)maxu(.,di/2,di). (22) 

di 

di/2 

The composition of max — moy operations names the algorithm, which is 
the classical zero-sum game max — min, where the rational opponent has 
been replaced by neutral providence. (|21| , p^ ) are consistent with ([T3D : after 
the judgment, they simply repeat the utility forever, so that the max — moy 
operations can be chained ad infinitum, no matter the end of round. Thus, 
the judgment can be arbitrarily postponed, without affecting strategy. If 
judgment times have an upper bound (e. g. the number of fates is finite), 
then all judgments can be postponed until a (collective) last judgment a time 
J G N, J > max J^, e. g. the maximum round duration. 

Relaxing the rule (|^), and taking dg as a parameter, the problem of fate 
management, i. e. finding optimal strategies, is self-similar under time- 
shifts, except for the parameter "renormalization" (as in statistical physics) 

(Do, J, do, (. . . d,), {p, u){.)) ^ {Dj, J - J, d„ 0, {p, u){... d,, .)). (23) 

Let xv be the characteristic function of \^ C B{D). The round providen- 
tial strategy is determined by (||) and 

p{., do, di/2) = p(di/2)x9B+(Do)(di/2). (24) 

The expected utility is computed with (|l^, |20D, from the last judgment back- 
ward in time: 

m(. . . dj_i, dj_i/2), m(. . . dj„i) . . . u{do, di/2, di), u(do, di/2), u(do), (25) 
e. g., for J = 3, and using (|24|) , 

m(- • •d3/2) = max ^ p(d5/2)M(. . . d3/2, d2, dg/a), 

M(do, di/2) = niax ^ p(d3/2)M(do, di/2, di, d3/2). 

d3/2eaB+(Di) 

M(do) = ^ ]9(di/2)M(do,di/2). 

di/2eaB+(Do) 
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3 Fate stochastic process 

For a given strategy, what is the presence density (of the die system in a 
subset of phase space)? What is the expectation of an arbitrary utility, for 
which the given strategy is not necessarily optimal? 



3.1 The Kolmogorov equation on expected utility 

Fate is a stochastic process, not only because it contains random events (the 
probability law p), but also random decisions, according to mixed strategies 
(the probability law P). For any causal process like (|^), the sequence of 
histories 



(do), (do,di/2), (do,di/2,di 



is a discrete Markov chain, for which classical results are available ||T2|, ch. 6], 



13, ch. 15], originating mostly from Brownian motion studies ch. 15]. 

The fate stochastic evolution equation, the Langevin equation, is just a 
random sum, obeying (^, ||): 

dj+i = dj + dj+i, 
^(dj+i = d| . . .dj,dj+i/2) = P(. . .dj,dj+i/2,dj + d), 
^(dj+i/2 = d| ...dj) = p(. ..dj,d). 

dj+i is a random source term, conditioned by history, according to the mixed 
strategies P,p. dj undergoes a strategy-driven Brownian motion as, for ex- 
ample, a charged Brownian particle driven by electrophoresis. 

The Chapman-Kolmogorov equation yields the probability of transition, 
or jump, from one state to the other, in one time step: 

do rx di) = P(di = d|., do) = 5^p(., do, di/2)P(., do, di/2, d). (26) 

Let P be a player's mixed strategy, possibly not optimal. From the von 
Neumann-Morgenstern theorem, twice applied. 



u{., do) = ^P(-, do, di/2) ^ P(., do, di/2, di)n(., do, di/2, di). (27) 

dl/2 di 
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Reversing the order of summation, using (p6|) and assuming that utility 
does not depend on events, but only on states, which is true in the 421 round, 
I obtain the Kolmogorov equation on the expected utility: 

do) = ^ do di)u(., do, *, di). (28) 

di 

(By hypothesis, u does not depend on *.) 

As opposed to the max — moy algorithm, (|^) does not produce any de- 
cision, but, given the mixed strategies P,p (effective through a), determines 
the expectation of any utility, for which P may not be optimal. 

Nevertheless, if P is optimal, from ( pT^ ) and ([T9|), there is an equality, 
between operators on m(., do, di/2, di): 

max = VP(,do,di/2,di). (29) 
Taking (|2|) into (^ returns (^. 

3.2 The Fokker-Planck equation on presence density 

I define the state fate ip = (dj)j=o,i... (fate with only states, not events). 
From ([26|) , 

ViiJ = (., do, di)) = a{., do rx di)P(^ = (., do)), (30) 
so that the sequence of past states 

(do), (do,di), (do,di,d2) . . . 

also is a Markov chain. 

Summing ( ^0|) over all state fates converging to the same state d at time 
j + I gives the presence density pj_|_i(d): 

Po(d) = 5d,do, (31) 
VjGN,p,+i(d) = 5^ P(^ = (do...d,)). (32) 

do...dj 

In the round, from ([Til) , Pj is stationary, as soon as j > J. 
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I assume that utility is a function of state and time only, less general than 
causal (|T^): 

u{...dj^)=uj^{dj^). (33) 



The end-of-round utility is indeed of the kind ([3^), because end-of-set ranking 
(see the rules) only depends on round results, not on intermediary states and, 
for the first player, the effective round duration. 

For all player's optimal (or rational) mixed strategy P derived from a 
utility of the kind ([33|) , 



^i(dj,dj+i/2,dj+i) = P(.,dj,dj-+i/2,dj+i), (34) 
aj{djr\dj+i) = a{.,dj r\ dj+i) (35) 

Taking (|3^, |3^) into (|28D allows to extend (|33D to all time (for the expected 
utility), by induction: 

yj eN,Uj{dj) = u{...dj). (36) 

The process (j, d^, dj+1/2) is Markovian. 

The consequence (|35D of (p3D, taken into (|32|), allows to express pj+i(dj+i) 
as a functional on pj-. 



the Fokker-Planck equation. 

As opposed to (|3^) , (p8|) does not need (|33|) . Nevertheless, with (|33|) , 
(p^), becomes 



(^j) = XI ^ dj+i)Mj+i(dj+i), (38) 



adjoint to (|37|). 

(|37| , |38|) are the evolution equations, adjoint to each other, linear, unsta- 
tionary, of presence density and expected utility. Their inputs are a player's 
mixed strategy and utility. 
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3.3 Computing result probabilities by duality 

Let JF = J^(B^{D),Q) be the space of numerical functions on B^{D), with 
the scalar product 

V(/, ^7) e ^, (/, ^7) = J2 /(dMd). (39) 

deB+{D) 

aj is an operator, a linear endomorphism on JF, fully determined by the 
Markovian matrix crj(do r> di). Its transposed operator is aj, of matrix 

o-*(di r\ do) = cTj {do r\di). 



In operator notation, (37, 38) become 



As cr* et aj are adjoint to each other, the expected utility follows a con- 
servation law: 

{uj,pj) = WjUj+upj) = {uj+uajpj) = {uj+i,pj+i), 

{uj,pj) = {uo,po) = uo{do). (40) 



The last equality is a consequence of (^I]). Given the player's mixed strategy 
P, (|Dp holds for any utility. 

The direct computation of {uj, pj) consists in solving for Pj{dj) the Fokker- 
Planck equation, which must be repeated, to complete the scalar product, 
at least for all d^ where Uj does not vanish. More shrewdly, {uj,pj) can be 
computed indirectly, as the r. h. s. of (|40|) : the Kolmogorov equation is 
solved only once for the expected utility at the trunk of the fate tree, or the 
initial expected utility. The indirect computation is faster than the direct 
computation, by a factor which is the cardinal of the support of m^ . The indi- 
rect computation benefits from the unicity of the fate tree, and the diffusive 
growth of the support of Uj. 

Moreover, to obtain the Kolmogorov algorithm from the max — moy al- 
gorithm, one merely has to replace, in (p^), the operator max appearing at 
the 1. h. s. of (^91), by the operator strat appearing at the r. h. s. of (^91). 
(These operators differ if P is not optimal.) The Kolmogorov equation is 
thus solved by a strat — moy algorithm. 



Here are examples of using the Kolmogorov equation and (WXl- 
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1. The probability of the result to be in C B^{D) (independently of 
time) is the initial expectation of the stationary utility Uj = xv- 

2. The probability of Dj^^ is the initial expectation of the utility Uj = 

^jdoXdB+(D-Dj^)- 

3.4 Analogy with linear transport theory 

The round is a linear transport phenomenon, with respect to the face variable. 
Face, expected utility, presence density, transition probability correspond re- 
spectively, in transport theory ||T^, to phase (position, velocity), importance 
ITSi , flux and cross section. Harris ||16| shows that a monokinetic particle 
population grown by branching (e. g. neutrons produced by nuclear fission) 
follows a Galton- Watson process. Similarly, in appendix 0, I discuss the 
Galton- Watson character of the first player's live dice population Dj. 

4 Simple optimal policies for one-goal utilities 

Taking for goal a unique combination d* G dB^{D), the utility is a binary 
Kronecker function 6d*,. (modulo an affine transform), and optimal strategies 
are simply constructed. 

4.1 The ratchet and Bernoulli policies 

I examine two first player's policies, with a one-goal utility: 

1. The Bernoulli policy consists in accumulating no die, unless the goal 
has been attained (then, all dice are accumulated); the cast sequence 
is a stationary Bernoulli process (a sequence of independent trials ter- 
minated by success or failure). 

2. The ratchet policy consists in putting aside as many dice as possible, 
contributing to the goal: 

V(jGN,j + l< J),d,+i = d*A(d,- + d,+i/2), (41) 
^j(di,dj+i/2,dj+i) = 5d,+i,d-A(d,+d,+i/2). (42) 
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(A is the infix notation of the minimum in the partially ordered space Z^, 
generalizing, in Lagrangian notation, the ensemble intersection fl.) 

The ratchet strategy towards d* is optimal, with respect to the d*-goal 
utility, if and only if p decreases in B~^{D). This means that as many dice as 
possible should be accumulated, in order to maximize the success probability 
at any future time. For unloaded dice, from (|l]), 

V,..«^,nd.e.e™^4^^1, (43) 

i. e. p decreases on B~^{F). The ratchet strategy is optimal if and only if 
D < F, strictly if and only if D < F. 

For example, with L' = 3<F = 6, J>l,d* = 421,di/2 = 651, the 
ratchet decision (to accumulate 1) is optimal, because p(421) < p(42) (it will 
be easier to obtain 42 than 421). With D = 3 > F = 2, d* = 211, di/2 = 222, 
the Bernoulli decision (to replay all dice) is optimal, because p{ll) = l/F"^ = 
2/8 < p(211) = = 3/8. With D = F = 2,d* = 21,di/2 = 11, both 

Bernoulli and ratchet decisions are optimal. 

A next player's maximum round duration is imposed. In case of a prema- 
ture success, he is in a dilemma, having to decide between equally unpleasant 
ways of breaking the goal, obtained too early. For F > 2, optimal decisions 
consist in replaying any one die; the number of pure optimal strategies is thus 
the number of distinct faces in the goal combination, at the power J — 1. If 
the goal is a brelan, then no dilemma exists. 

4.2 Optimal one-goal strategy result probabilities 

For any strategy, I consider the probability to obtain any result, e. g. Ill 
after three casts. According to section |373| , this probability is the initial 
expected utility, determined by the Kolmogorov equation and the final con- 
dition of a Kronecker utility on the result. This probability depends on the 
player i = 1,2 (first or next), the (renormalized) maximum round duration 
Ji, the player's mixed strategy P, the delay j, and the result d: 

p,( Ji, P,j, d), < J < Ji < J, d G 5+(D). (44) 

The set of result probabilities, for all possible pure strategies and {D, F, J) = 
(3,6,3), is (much larger than the fate tree, itself very large and) too large 
to be extensively listed. Thus, I will work on a reduced strategy subset, for 
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which a reasonable choice is the set of optimal one-goal strategies, for all pos- 
sible goals. As far as the goal determines the optimal strategy, the variable 
P in (|4^ is simply replaced by the goal d*: 

Ji, d*,j, d), < J < Ji < J, (d, d*) G B+{DY (45) 

which looks like the Markovian matrix of section |373| , except that d* is not 
actual, but contemplated. There are diagonal (d = d*) and non-diagonal 
result probabilities. 

For the first player, the optimal one-goal strategy is unequivocally defined 
by the goal {D < F: the ratchet) and the function pi is defined everywhere. 
This in not true for p2, because of dilemmas. However, next player diagonal 
probabilities are unaffected by dilemmas, so that p2 is defined on the diagonal, 
d = d*; it is even defined for all (d*,d), if and only if d* is a brelan, since 



brelans do not produce dilemma, as noticed at end of section ^ 
Here are a few properties of the functions pi: 

p,(0,d*,0,d) = 5d.,d, (46) 

p,(J,0,j,0) = 5,,o, (47) 

p,(l,d*,l,d) = p{d), (48) 

p,(J,d*,j,d) = 0,j<J,dVd, 

Pi(J,d,j,d) = pi{j,d,j,d)J<J, 

P2{J,d*,j,d) = 0,j<J, (49) 

J 

J2 $^Pi(J,d*,j,d) = 1. 

de9B+(|d*|) j=0 

P2U,Def,j,d) = 1. 

dg<9B+(|d*|) 

Let the cumulative diagonal probability be 

J 

s,{J,d) = Y,MJ,d,j,d). (50) 

i=i 

Because of the next players' round duration condition 



VJ > 1, si( J, d) > S2( J, d) = p2{J, d, J, d) > pi{J, d, J, d). 
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To reduce the Pi computational domain, I use invariance with respect 
to face permutations (for unloaded dice). Firstly, diagonal probabilities de- 
pend on only one combination. As in (|l]), two combinations are equivalent, 
modulo the functions d i— > pj(J, d, j, d), for all (i, J, j), if and only if their 
occupation numbers (Lagrangian components) form the same combination, 
e. g. 441 ~ 655. With {D,F) = (3,6), the quotient set contains three 
classes: that of brelans {3 111), that of sequences {3 123)[|, that of pairs 
{3 112). Secondly, non-diagonal probabilities depend on a couple of combi- 
nations. Two couples of combinations are equivalent, modulo the functions 
(d*, d) pi{J, d*,j, d), for all {i, J,j), if and only if their couples of occu- 
pation numbers form the same combination, e. g. (421,442) ~ (321, 211). A 
face permutation transforms a next player's optimal one-goal strategy into 
another, possibly different if the goal is not a brelan. 

Taking into account ( |^ ) and face permutation invariance, the result prob- 
abilities (^) are computed, for {D, F, J) = (3, 6, 3), by applying strat — moy 
on optimal d*-goal strategies and d-Kronecker utilities. As a consequence 
of self-similarity (p3D , the probabilities after the initial time (Ji < J), are 
obtained as intermediary results in the computation of a priori probabilities 
(Ji = J). The results are presented in the probability charts H, 0, ||, ||, |1^ 
(appendix |C|), which do not fill more than a few pages thanks to the exten- 
sive use of face permutation invariance and other properties (|4^ . . . ). There 
are 31 classes of three-die combination couples (including the three diagonal 
classes). 



5 Goal identification programming 

I will propose heuristic policies, based on the global maximization of expected 
utility, with respect to the subset of optimal one-goal strategies, for which 
result probabilities were obtained in the last section. 

5.1 Motivation: bounded complexity 

The max — moy backward induction algorithm is optimal, short, but the 
number of numerical operations per time step, already large for {D, F, J) = 
(3,6,3), is unbounded as a function of the maximum round duration J. 
Information theory [1^, |^ teaches that a message will be transmitted faster 

do not mean that all combination in the class of sequences is a sequence. 
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by a specialized code, max — moy backward induction is slow, for the general 
reasons that it is unspecialized (and optimal). 

To speed-up policy, possibly at the expense of brevity and optimality, 
specialization is necessary. For example, consider the game of Nim f^, § 1.3]: 
besides max — moy backward induction, a stratagem is found, based on con- 
gruence, producing optimal strategies, with a bounded number of operations 
per time step. The ratchet {D < F) would be a stratagem of 421, if only the 
goal were known. 

I propose to identify the goal, rigorously, by considering not only the 
utility, but also the result probabilities (^51), obtained in section ^. I will 
obtain goal identification heuristic policies, that may be considered as quasi- 
Markovian, from the remark following (^). Roughly, they transfer the com- 
plexity of max — moy backward induction to the result probabilities, with the 
advantage that the latter can be compiled once for all (and the inconvenience 
that they must be remembered). 

For a one-goal utility, goal identification is simple. For a constant utility, 
as well: any goal is optimal. Difficulties are thus with utilities somewhere 
between peaked and flat, "fuzzy", e. g. with peaks of about the same height, 
playing the roles of attractors, that one has to choose between.^ 



5.2 Reduced horizon 



I consider a time and state dependent utility, as in ( [3^ , |36|), in a round of 
maximum duration J. do is the state at time jo < J- I deflne the "evaluation 
function", 

J-jo 



do) = rnax pi(J - Jq, d*, j, d*)uj„+j(do + d*), (51) 



]=0 



where j is the renormalized time and Dq = D — |do|. Evaluation functions 
are often used in stage game (chess, othello, checkers . . . ) programming, but 
they are usually deflned empirically, unlike (pT|), which is probabilistic. 

To take into account serendipity - that a result other than the goal may 
be not so bad, after all - (|5lD is improved: 

J-jo 

^0 (do) = ^ max J2 - ^0, d*, J, d)uj^+j{do + d), (52) 

j=0 dedB+(Do) 



*Like Buridan's donkey, starving from hesitating between bushels of oats and water. 
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which cannot be used for next players, because of dilemmas. For all dg G 
dB~^{D), considering (^), the evaluation functions ^2|) simply return 
the utility. 

max — moy backward induction is particularly slow, because it needs to 
completely analyze the round even before its first decision. Hence the idea 
that short-sighted policies may be faster. At time jo G N, a horizon h E N 
may be chosen, such that ji = jo + h < J, and the round is virtually 
terminated at j'l, taking for ersatz utility the evaluation function u*^ given 
by (|5lD or (^), depending on the serendipity bit s G {0, 1}. With ji = J — I, 
considering pS]), (|52|) reproduces the deepest max — moy iteration, so that 
an optimal strategy is generated. 

I will further examine h = 0,1. With h = 0, the goal is found by max- 
imizing u*^, independently of the first event. With h = 1, as there is no 
interest in thinking before casting the dice, the decision di is rather taken 
after the first event di/2, according to 

niaxu*^%i(di). (53) 

In case of many optimal decisions in ([5^), the corresponding states, written 
as increasing Lagrangian lists, e. g. 124, are discriminated according to the 
lexicographic order (only pure strategies are generated). In case of many 
optimal goals in (|5ll) or (|52|) , we need not discriminate between them, and 
the policy reproduces the human character of duplicity. Dilemma implies 
duplicity, but the converse is false. 

5.3 Dynamic programming and goal revision 

The strategy may be revised to take into account new events, which is an 
instance of dynamic programming ||T8| or belief revision ||T^, realizing a feed- 
back of fate on strategy. By self-similarity of the round, a policy may be 
applied at any time, with suitable parameter renormalization. Self-similar 
revision based on the max — moy backward induction policy would just con- 
firm the optimal strategy, computed a priori: it is therefore useless. Only 
fallible policies are worth revising. 

A heuristic policy of horizon h > 1 forecasts, at any given time, only the 
next h decisions. Thus, it must be run with the period at least h. The revised 
serendipitous goal identification policy of horizon h is optimal in its last h 
decisions. The goal identification policy with h = does not require revision 
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Table 1: 123 one-goal utility 



uor = 0.22811 


Uo/UOr 


policy 


player 


horizon 


serendip. 


first 


next 








1 


0.57858 





1 


1 




1 





1 


0.57858 


1 


1 


1 




max — moy 


1 


0.57858 



and is very simple (short and fast) . It may be the only rational policy, simple 
enough for unaided human players in normal game conditions. 



5.4 Policy benchmark and interpretation 

For {D, F, J) = (3, 6, 3), I consider a few increasingly fuzzy stationary utili- 
ties: 

1. u = 5i23, a one-goal utility, 

2. u = 5i23 + ^224 + ^345, a three- goal utility, 

3. u = t, the transfer function defined by table || in appendix, 

4. the sum of faces. 

These utilities are unrealistic, in the sense that they may not be possible 
within a real 421 set (see section |27^) . I consider the policies: max — moy 
backward induction, and the four goal identification policies {h,s) G {0, 1}^; 
the h = policies are without revision. 

From the final utility, on the leaves of the fate tree, every policy yields 
a pure strategy, and its initial expected utility uo, on the trunk, is obtained 
by solving the Kolmogorov equation exactly, with the strat — moy algorithm. 
Optimality is defined as the ratio of the expected utility, over the first player 
optimal expected utility Wor. The numerical results (approximated by deci- 
mal numbers) are copied from ||2^ into the tables |l], ||, ||, ||. 

Table || confirms that for a one-goal utility, all goal identification poli- 
cies are by definition optimal. Compared to the first player, next players 
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Table 2: 123, 224, 345 three-goal utility 



uor = 0.32805 


Uo/Uor 


policy 


player 


horizon 


serendip. 


first 


next 








0.73037 


0.43734 





1 


0.73037 




1 





0.97777 


0.47746 


1 


1 


0.98657 




max — moy 


1 


0.49152 



Table 3: utility = transfer function 



UQr = 3.7467 


Uo/Uor 


policy 


player 


horizon 


serendip. 


first 


next 








0.90834 


0.68812 





1 


0.90834 




1 





0.87962 


0.68991 


1 


1 


0.99634 




max — moy 


1 


0.77663 


Table 4: utility = sum of faces 


Uor — 14 


Uq/ Uor 


policy 


player 


horizon 


serendip. 


first 


next 








0.94194 


0.92599 





1 


0.96418 




1 





0.75 


0.85875 


1 


1 


0.99900 




max — moy 


1 


0.97321 
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are handicapped, but less with a fuzzier utility. The numerical results show 
a positive contribution of serendipity, much greater with the greater hori- 
zon and revision. The contribution of horizon and revision is positive with 
serendipity. Without serendipity, the contribution of horizon and revision is 
positive for peaked utilities, negative for fuzzy utilities (||, U). 

I take advantage of this effect to give a (less fuzzy) definition of fuzzi- 
ness: a utility is fuzzy if and only if introducing horizon and revision with- 
out serendipity contributes negatively to its expectation. Thus, I have con- 
structed fuzzy utilities, for which introducing horizon and revision decreases 
the expected utility, even though it is more complex. The response of ex- 
pected utility with respect to complexity is non-increasing (this effect com- 
pares, in electricity, with a negative resistance). 



6 Conclusions 

The mathematics of fate in 421 leave as the only unsolved difficulty "bifur- 
cations", that maximizing the expected utility does not always determine a 
unique decision, as in next players' dilemmas. Here is a toy example: a game 
with three players, P, A, B. If P says white, then A gives one euro to B; if P 
says black, then B gives one euro to A. P earns nothing anyway; A, B take 
no decision. Maximizing P's expected utility does not determine its decision. 
Introducing a mixed strategy amounts to consider P as a random generator, 
with unknown probabilities. A classic postulate of statistical theories is to 
maximize the entropy or missing information ||lO| , |l7| , which here sets the 
probabilities of either outcome to 1/2. Are the postulates of mixed strategy 
and maximum entropy so easily acceptable? We cannot exclude hidden de- 
terminism or bias in P. For example, P may always choose the first answer 
in the lexicographic order (black), or P may have a secret agreement with A 
to share his gain. 



Bounded complexity, similar to bounded rationality in [pT| , motivates 



heuristic policies, where characters close to actual human behavior are found. 



in agreement with |^]. These characters are fate, dilemma, goal identifica- 
tion and revision, restricted horizon, serendipity, duplicity and panic. When 
the policy belongs to an organization, we are in management. When an in- 
dividual decides for himself, we are in psychology. For example, the same 
mathematical effect is behind counterproductive management or panic. 
Goal identification consumes a bounded number of operations per time 
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step, whatever the round duration, because it does not resolve all decisions 
in the fate tree, but only those which are compatible with the present state, 
and before the horizon. Goal identification is not generally optimal, as op- 
posed to a common assertion in business courses. Only max — moy backward 
induction, which has no goal, just like random playing, is generally optimal. 
In the round, the ratchet stratagem allows the immediate translation from 
goal to decision. I used probability theory as the logic of goal identification, 
a la Jaynes ||lT]. Complexity hides in the result probabilities, to be compiled 
before playing, kind of training. 

Depending on complexity resources and utility, policies may be variably 
applicable or good. Starting from a given policy, one may increase optimality, 
by modifying its characters or the utility: this is the task of human resources 
management, when the policy is that of an individual taking decisions for 
a company, a manager. The short-sighted manager {h = 0) gets hardly 
any help from serendipity. The unserendipitous manager should avoid fuzzy 
utilities and favor precise goal assignments. I obtain examples of counterpro- 
ductive management: with a fuzzy utility and no serendipity, goal revision 
dramatically reduces the optimality. The role of serendipity was pointed out, 
on purely qualitative ground, by N. Wiener, about scientific and technical 
invention |^. The present work also pertains to Wiener's cybernetics. 

Rationality can be further reduced. At the extreme, the fool manager 
can be trusted only for a flat utility. The study of irrational or illogical but 
actual behavior is the task of sophistry It may be quite useful in game 
practice, to produce best responses. 

I thank researchers of the GREQAM in Marseilles, for fruitful discussions. 



A The (tentative) rules of 421 



I define the game, from oral tradition and [^, |26[. The hardware consists 
of three dice and eleven tokens, initially in a pot. There are two or more 
players who can always see the positions of dice and tokens. 

In the first part of the game, the charge, players get tokens from the pot. 
In the second part of the game, the discharge, players get tokens from each 
other. A player wins when he gets no token during the charge (many players 
may thus win), or when he first gets rid of his tokens during the discharge. 

The charge or discharge is a sequence of sets. In every set, each player 
at his turn plays a round against the dice, while the others wait. The active 
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Table 5: token transfer function 



highest combination 


token number 


421 


10 


111 


7 


fUJffJ^l 


/ 


sequence 


2 


other 


1 



player casts the dice up to three times; after every cast, he can put aside 
any number of dice, thus accumulating a combination. Next players must 
cast dice as many times as the first player.0 End-of-round accumulated com- 
binations, obtained by all players in the set, are ranked in the hierarchic 
order 

421 y 111 y 611 y 666 y 511 y 555 y 411 y 444 y 311 

y 333 y 211 y 222 y 654 y 543 y 432 ^ 321 ^ 665 ^ . . . 221, (54) 

where y means 'higher than'. The combinations, implicit in (|54|) , are ordered 
as the numbers formed by their faces in a decreasing sequence: e. g. 655 >- 
654. The dominant combination 421 and the dominated combination 221, 
known as "nenette", differ only by one die. /// is the /-brelan, /II is the 
/-pair (/ 7^ 1), 654,543,432,321 are the sequences. 

At end of set, the last0 player who has got the lowest combination gets the 
number of tokens determined by table ^, e. g. if the highest combination is 
411, then the last player with the lowest combination (whatever it is) gets 4 
tokens. During the charge, tokens are taken from the pot, if possible. When 
the pot is empty, the discharge begins, and tokens are now taken from the 
player who has got the highest combination. 



^The order of players in the set matters, but I could not find definite rules for its 
determination. 

^The adjective 'last' is my own suggestion for automatic tie-breaking. 
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B A Galton-Watson process in the 421 round 

Taking the genealogic point of view, each die is considered as an individual, 
dying after being cast, either without a child, in case of accumulation, or 
with a single child (itself indeed). The child number being lower than one, 
the number of live dice Dj (section ^73|) decreases in time. Moreover, the 
population becomes extinct after J casts (or sooner). 

A Galton-Watson process is obtained when the offspring of each in- 
dividual is independent of others'. With an optimal d*-goal strategy, the 
dice dying without children have their faces in d*, but the converse is not 
true. For example, with d* = 221, di = 211, J > 1, the two dice 11 have 
correlated offspring: one has a child if and only if the other has none. Dice 
have independent offspring if and only if d* is a brelan and the player is first. 

I apply the Galton-Watson theory ||12|, §6.2] to obtain the probability law 
of Dj, for an optimal d*-goal strategy, where d* = Dep is the F-brelan. Dice 
are indexed hy d = 1 . . . Dj . Let G {0, 1} the number of children of the 
die indexed by d. 

D, = J2 Zd- (55) 

d=l 

The Zd are random variables, with the same law = P(^d = i), of generating 
function 

g{z) = {z^") =qo + qiZ, go = gi = 1 - go. 
The Zd are always independent if and only if d* is a brelan and the 



player is first. When this is true, from (|55|), the generating function of Dj, 
conditioned by -Dj-i, is 

{z''^\D,^, = d)=g{zY. 
The generating function of Dj is thus determined by 

go{z) = z^,gj{z) = 1, 



D 



V(j, 1 < J < J),g,{z) ^ {z^^) = = d)V{D,^i = d) 

D 

Y,'P{Dj-i = d)g{zY = g,^rog 



d=0 

D 



d=0 
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By induction, 

9j = 90 ° 9°^- 

The composition powers of the affine function g are 

g°i(z) = l-ql+ q{z. 

Therefore 

9A^) = (l - qi + qiz)'' , 
V{D,=d) = (1 - gj)''- V- (56) 

Dj follows a binomial law, directly obtained by considering that a die 
dies when accumulated, or stays alive, with the probability qi per time step, 
independently of others: a Bernoulli process is obtained, with the law (|56D . 
The interest of considering a Galton- Watson process is in the analogy with 
branching processes ||16| . 



C Realization with mathematica 



The present article is supported by ||2^, an open source software and data 
base in the mathematica language ||2^, which, like LISP, is interpreted and 
allows functional and recursive treatments on arbitrary expressions, equiva- 
lent to trees. The mathematica frontend allows literate programming ||2^ in 
the form of notebooks, gathering live code, outputs and comments, within a 
tree structure, that can be unfolded at will. 

Combination manipulation differs slightly from list manipulation (since 
order does not matter in combinations) or ensemble manipulation (since repe- 
titions are allowed in combinations). A tool box is developed. The numerical 
parameters {D, F, J) are arbitrary, which realizes a scalable model, invalu- 
able for development. Fate trees are created recursively. All fates converging 
to the same state at the same time are merged by indexing, so that the 
size grows only linearly with the depth J and remains easily manageable for 
{D,F,J) = (3,6,3). In exchange, the computing time is increased and the 
history is lost, which allows to treat only time and state dependent utilities 
(as required in the 421 set). 
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Starting from the leaves of the fate tree, where utility is grafted, opti- 
mal strategies and expected utilities are build recursively, according to the 
max — moy algorithm. A utility-strategy tree is finally obtained, from which 
the strategy can be extracted, then piped into the strat — moy algorithm, a 
variation on max — moy, solving the Kolmogorov equation. 

max — moy produces the expected optimal one-goal strategies, Bernoulli 
or ratchet, depending on D < F, and dilemmas. The result probabilities 
are computed, saved, and many properties are checked systematically. Some 
result probabilities are checked by Monte Carlo simulations, with success. 
The charts H, |^, ||, ^ |10| are generated automatically. There is very little 
room for errors, and if there are any, they are traceable. 

The goal identification heuristic policies are realized. Their wrong deci- 
sions are pointed out. They are exactly evaluated with strat — moy, which 
is very slow, since it requires the computation of every heuristic decision in 
the fate tree, according to an algorithm actually longer and slower, for one 
decision, than the simple maximization in max — moy. Obtaining the truth 
about heuristic policies is a lengthy task. 



Probability charts player's guide 

pi, p2 mean first or next players. In every box of a diagonal probability 
chart stands a column of the probabilities, ordered from top to bottom by 
growing delay, to obtain the goal written at head of line. 

In every box of a non-diagonal probability chart, stand two columns: at 
left, from top to bottom, the goal and the result; at right, the probabil- 
ities, ordered from top to bottom by growing delay, to obtain the result, 
with the goal in mind (and taking optimal decisions as determined by the 
ratchet). Moreover, for easy access, the couples (goal, result) are represented 
in a square array, where heads of lines and columns are the respective rep- 
resentatives of goal and result, modulo face permutations (section |4^). The 
three-die representative 3X3 array is spread onto the three charts ||, ^ [1^, 
one for each goal class. 

Here is an example for using non-diagonal charts. Let the goal be 641 
and the result 652. The representatives of 641 and 655 are, separately and 
respectively, 123 and 112. (Representatives are chosen so as to minimize the 
sum of their faces.) The representative of the couple (641,655) is (123, 144). 
123 takes us to chart |1^ (the third line of the representative square array), 
whence 112 takes us to the second column, (123, 144) to the third row, where 
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finally are the probabilities to obtain, with the goal 641, the result 652, after 
one, two or three casts. 
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Table 6: diagonal result probabilities 



pi p2 

^ = 0.1667 ^ = 0.1667 

6 6 

0.1389 ^ = 0.1667 



5 _ n 1 T Q Q 1 
36 



pi p2 

= 0.02778 = 0.02778 

]_]_ 36 36 

= 0.06559 = 0.07022 

1296 1296 

^ = 0.05556 ^ = 0.05556 

18 18 

3^ - 0.108 = 0.1173 



324 162 



pi p2 

= 0.00463 = 0.00463 

216 216 

111 JJJ^ ^ 0.0239 ^ 0.02467 

46656 46656 

^ 0.04625 513991 ^ 0.051 

10077696 10077696 

^ = 0.01389 ^ = 0.01389 

72 72 

112 = 0.05517 = 0.05748 

2592 2592 

23681 _ 0.08459 ^6903_ = 0.0961 



279936 279936 

= 0.02778 = 0.02778 

36 35 

123 = 0.08758 = 0.09221 

2592 2592 

21043 ^ 0.1128 24631 ^ Q_i32 

186624 186624 



REALIZATION WITH MATHEMATICA 



Table 7: first player's non-diagonal result probabilities 

1 

1 ^ =0.1667 
1 ^ ^ 

2 ^=0.1389 

35 



11 



12 



11 



12 



11 
22 



12 
33 

12 
1 1 



1 



= . 02778 



36 

25 =0.01929 



1296 



1 

36 

^=0.01235 

81 



1 

36 
35 
648 



=0.02778 
= . 05401 



11 
23 

11 
12 



0.02778 12 
34 



12 
13 



^ =0.05556 

18 

^^ = . 03858 

648 

^ =0.05556 

18 

^^ = 0.08488 

648 

^ = . 05556 
18 

^=0.02469 

81 

^ =0.05556 

18 

^^ = 0.06636 

648 
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111 
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216 
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46656 
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10077696 



:0. 00463 
:0. 002679 
:0.00155 



111 
223 



111 
122 



111 
112 



112 

1 

72 

125 
15552 

15625 
3359232 




3359232 
1 

72 

605 
15552 
207025 
3359232 



0.01389 

:0 . 008038 
:0. 004651 

0. 01389 
0. 01768 
:0 . 01693 

0. 01389 
0. 0389 
0. 06163 



111 
234 



111 
123 



123 



1 

36 

125 
7776 

15625 
1679616 

1 

36 

275 
7776 

56875 
1679616 



:0 . 02778 
:0 .01608 
:0. 009303 

:0. 02778 
:0. 03537 
:0 .03386 



a; 
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o 



o 



s 
I 
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112 
334 



216 



:0. 00463 



112 

=0.001372 

333 729 

:0. 0004064 



216 



:0 .00463 



112 
233 



19683 * 1 1 2 

^ =0.00463 -"-^^ 



216 
112 

112 =0.009216 

222 23328 

20605 =0 . 008178 

2519424 1 1 2 



223 



112 205 =0 . 01758 
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